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DESCRIPTION 

IMAGE PROCESSING APPARATUS, IMAGE PROCESSING METHOD, AND 
RECORDING MEDIUM 

f* Technical Field 

P 

O The present invention relates to an image processing apparatus, an image 

M 

p processing method, and a recording medium, and more specifically to an image 

y processing apparatus, an image processing method, and arecording medium which are 

13 suitable for the case of synthesizing a hair image expressed by computer graphics with 

in 

Ji; a three-dimensional shape image of a head part is removing a hair. 

E3 

ru 

Background Art 

Conventionally, as a method for expressing human hair by computer graphics : 
(hereinafter abbreviated as CG), it is useful to take a method in which an entire hair 
style is approximated to a polygon model, as shown in FIG. 1 A, and a texture of hairs 
is adhered to the model, or a method in which rendering is performed with use of an 
anisotropic reflection model which has different reflection intensities respectively with 
respect to light source positions or viewpoint positions. 

These methods are described in "Hair expression using an anisotropic reflection 
model" by Yamana and Suenaga, Singakugihou, PRU87-3, pp. 115 to 120, 1989. In 



these methods, expression of a hair style is just the shape of a polygon model itself, 
and the hair style can be changed by changing the shape of the polygon model is 
realized. 

A proposal has been made for a method of expressing hair by a particle system. 
In the particle system, when a large quantity of particle objects are dealt with, 
characteristics such as weight and the like of each particle are given as parameters, and 
particles are generated under a certain rule. Thus, a large quantity of particle objects 
can be expressed easily. 

Meanwhile, the present applicant has proposed a method of approximating 
every one of hairs to a curve, as shown in FIG. IB, in Japanese Patent Application No. 
10-164545, for example. This method is successful in natural expression of a hair 
style image. 

A method of designing a hair style by editing together a plurality of hairs 
approximated to a curve is described in "Development of a hair design system using 
tuft models" by Kishi, Saegusa, and Morishima, in Shingakugihou, MVE97-102, pp. 
67-74, 1998. 

It is difficult for the method of approximating the entire hair style to a polygon 
model to express a complicated hair style. In this method, movements (flows) of hair 
cannot be expressed. 

On the other side, the method of expressing hair by a particle system and the 
method of approximating every one of hairs to a geometric figure can express any 



arbitrary hair style and movements of hair. However, in the method of expressing hair 
by the particle system, parameters for particles expressing hairs must be controlled to 
control the hair style, so that intuitive control is difficult. It is therefore difficult to 
generate an intended hair style. 

In the method described in "Development of a hair design system using tuft 
models" by Kishi, Saegusa, and Morishima, in Shingakugihou, MVE97- 102, pp. 67-74, 
1998, an intended hair style can be generated relatively easily. This method, however, 
has a problem in that hair cannot be changed to intended hair styles corresponding to 
a plurality of different head shapes although hair corresponding only to one fixed head 
shape can be changed to an intended hair style, 

Disclosure of the Invention 

: The present inventionhas been made in view of the actual situation as described 
above and has an object of changing a hair style consisting of hairs every of which is 
expressed by CG, so as to correspond to plural of different head shapes. 

The present invention proposed to achieve the above obj ect provides an image 
processing apparatus for synthesizing a hair image with a three-dimensional shape 
image of a head part, to generate a hair-style-matched image, comprising: storage 
means for storing a hair-style data piece; read out means for reading out the hair-style 
data piece stored in the storage means; mapping means for mapping a hair data piece 
corresponding to a hair and contained in the hair-style data piece read out by the read 



out means, at a predetermined position on the three-dimensional shape image of the 
head part; and generation means for generating the hair-style-matched image, based 
on a mapping result obtained by the mapping means. 

As the hair data piece, it is possible to use three-dimensional curve data which 
is constructed by a plurality of control points. 

As the hair data piece, it is possible to use polygon data which is constructed by 
a plurality of polygons. 

The storage means stores the hair-style data piece constructed by plural hair 
data pieces arranged on a two-dimensional array. In particular, the storage means 
stores the hair-style data piece constructed by plural data pieces arranged on a 
two-dimensional array corresponding to a projected image obtained by projecting a 
three-dimensional hair style expressed by a columnar coordinate system, on a 
two-dimensional coordinate system. 

The apparatus according to the present invention further comprises interpolation 
means for interpolating the hair data piece at the predetermined position, with use of 
a hair data piece close to the predetennined position. The interpolation means . 
interpolates the hair data piece, based on group information contained in the hair data 
piece close to the predetermined position. 

.The mapping means includes coordinate conversion means for converting the 
hair data piece expressed by a first coordinate system into data of a second coordinate 
system by which the three-dimensional shape image of the head part is expressed. 
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The coordinate conversion means converts the hair data piece expressed by the 
first coordinate system into data of a local coordinate system having an origin on a 
surface of the head part, and thereafter converts the data into data of the second 
coordinate system by winch the three-dimensional shape image of the head part is 
expressed. 

When the hair data piece expressed by the first coordinate system is converted 
into the data of the second coordinate system by which the three-dimensional shape 
image of the head part is expressed, the coordinate conversion means executes at least 
one of coordinate axis rotation and origin shift, based on a random number. 

Also, the present invention provides an image processing method for 
synthesizing a hair image with a three-dimensional shape image of a head part, to 
generate a hair-style-matched image, comprising: a storage step of storing a hair-style 
data piece; a read outstep of reading out the hair-style data piece stored by processing 
in the storage . step; -a- mapping . step of mapping a hair data piece corresponding to a 
hair and contained in the hair-style data piece read out by processing in the read 
outstep, at a predetermined position on the three-dimensional shape image of the head 
part; and a generation step of generating the hair-style-matched image, based on a -. 
mapping result obtained by processing in the mapping step. 

In the processing in the storage step according to the present invention, the 
hair-style data piece constructed by plural hair data pieces arranged on a 
two-dimensional array is stored. 



6 

Also, in the processing in the storage step, the hair-style data piece constructed 
by plural data pieces arranged on a two-dimensional array corresponding to a projected 
image obtained by projecting a three-dimensional hair style expressed by a columnar 
coordinate system, on a two-dimensional coordinate system, is stored. 

The method according to the present invention further comprises an 
^ . interpolation step of interpolating the hair data piece at the predetermined position, 

?I with use of a hair data piece close to the predetermined position: 

M 

53 In processing in the interpolation step, the hair data piece is interpolated, based 

O 

W on group information contained in the hair data piece close to the predetenriined 

L- position. Li' * • - 

rv : , The, processing in the mapping step includes a coordinate conversion step . of 

p converting the hair data piece expressed by a first coordinate system into data of a 

k 

second coordinate system by which the three-dimensional shape image of the head part 
is expressed • ■ ... 

In processing in the coordinate conversion step, the hair data piece expressed 
by the first coordinate system is converted into data of a local coordinate system 
having an origin on a surface of the head part, and thereafter, the data is converted into 
data of the second coordinate system by which the three-dimensional shape image of 
the head part is expressed. 

When the hair data piece expressed by the first coordinate system is converted 
into the data of the second coordinate system by which the three-dimensional shape 



image of the head part is expressed, at least one of coordinate axis rotation and origin 
shift is executed, based on a random number, in processing in the coordinate 
conversion step. 

Further, the present invention provides a recording medium which records a 
computer-readable program for image processing of synthesizing a hair image with a 
three-dimensional shape image of a head part, to generate a hairr style-matched image, 
the program comprising: a storage step of storing a hair-style data piece; a read outstep 
of reading out the hair-style data piece stored by processing in the storage step; a 
mapping step of mapping a hair data piece corresponding to a hair and contained in the 
hair-style data piece read out by processing in the read outstep, at a predetermined 
: position on the, three-dimensional shape image of the head part; and a generation step 
of generating the hair-style-matched image, based on a mapping result obtained by 
processing in the mapping step. 

- In the processing in the storage step, the hair-style data piece constructed by 
plural hair data pieces arranged on a two-dimensional array is stored. 

In the processing in the storage step, the hair-style data piece constructed by 
plural data pieces arranged on a two-dimensional array corresponding to a projected 
image obtained by projecting a three-dimensional hair style expressed by a columnar 
coordinate system, on a two-dimensional coordinate system, is stored. 

According to the present invention, the program further comprises an 
interpolation step of interpolating the hair data piece at the predetermined position, 
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with use of a hair data piece close to the predetermined position. 

In processing in the interpolation step, the hair data piece is interpolated, based 
on group information contained in the hair data piece close to the predetermined 
position. 

The processing in the mapping step includes a coordinate conversion step of 
converting the hair data piece expressed by a first coordinate system into data of a 
second coordinate system by which the three-dimensional shape image of the head part 
is expressed. 

In processing in the coordinate conversion step, the hair data piece expressed 
by the first coordinate system is converted into data of a local coordinate system 
having an origin on a surface of the head part, and thereafter, the data is converted into 
data of the second coordinate system by which the three-dimensional shape image of 
the head part is expressed. 

< - When the hair data piece expressed by the first coordinate system is converted 
into the data of the second coordinate system by which the three-dimensional shape 
image Of the head part is expressed, at least one of coordinate axis rotation and origin 
shift is executed, based on a random number, in processing in the coordinate 
conversion step. 

In the above-described image processing apparatus and method according to the 
present invention, hair-style data is stored, and the stored hair-style data is read out. 
Hair data corresponding to a hair and included in the read outhair-style data is mapped 



at a predetermined position on a three-dimensional shape image of a head part. Based 
on the mapping results, a hair-style-matched image is generated. The stored hair-style 
data is constructed by plural pieces of hair data arranged on a two-dimensional array. 

Further other objects of the present invention and specific advantages which 
will be obtained from the present invention will be more clearly understood from the 
following description of embodiments. 

Brief Description of the Drawings 

FIGS. 1 A and IB are views showing an example of conventional hair 
expression. 

•t FIG. 2 is. a block , diagram showing a structural example of a hair synthesis 
system to which the present invention is applied. 

FIG. 3 is a flowchart explaining operation of the hair synthesis system. 
: FIG. 4. is a view for explaining the operation of the hair synthesis system. 

FIG. 5 is a block diagram showing a structural example of processing in the 
hair-style mapping processing section shown in FIG.: 2. 

FIG. 6 is a flowchart for explaining processing in the hair-style mapping 
processing section. 

FIGS. 7 A and 7B are views for explaining hair-style data in which hair data is 
two-dimensionally arrayed. 

FIG. 8 is a view for explaining mapping position calculation in the step S13 
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shown in FIG. 6. 

FIG. 9 is a view for explaining hair data interpolation processing in the step S 1 5 
. shown in FIG. 6. 

FIGS. 10A and LOB are views for explaining coordinate conversion in the step 
S16 shown in FIG. 6. 

y, • FIG. 1 1 is a view showing a data structure of the hair-style database shown in 
|. FIG. I. 

63 FIG. 12 is a view for explaining interpolation processing using group 

E3 

infonnation included in hair data. 

p FIGS. 13A and 13B are views for explaining coordinate axis rotation in 

11 

jy, : coordinate conversion. ... . . . 

m 

P FIGS. 14A and 14B are views for explaining coordinate axis shift in coordinate 

conversion, . 

FIGS. 15 A and 15B are views for explaining of mapping to the three- 
dimensional model of a head part with an arbitrary changed. 
, FIG. 16 is a view showing a columnar coordinate system when hair-style data 

is generated.. 

FIG. 17 is a view showing a structural example of a personal computer which 
realizes an image processing apparatus. 



Best Mode for Carrying out the Invention 
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A hair synthesis system to which the present invention is applied will be 
explained with reference to FIG. 2. This hair synthesis system comprises: an image 
pickup device 1 which picks up an image of a neck part (the part above the neck, 

'., . including the neck) of a person; an image processing device 2 for. generating a 
three-dimensional shape model of the neck part, based on the image of the picked-up 
neck part, synthesizing a CG of an arbitrary hair style, and generating a 

55 hair-style-matched image; and a display device 9 for displaying the generated . 

B hair-style-matched image. If this hair synthesis system is installed in a beauty salon, 

O 

yj a barber's shop, or the like, it can be used to assist customers to select hair styles or so. 

UJ 

* - The image pickup device 1 of the hair synthesis system combines a 

H three-dimensional shape measurement device such as a stereo camera, a range finder,, 

Ul 

P or the like, and a two-dimensional image input device such as a CCD (Charge Coupled 

fU 

Device) or the like. In case of a stereo camera, the three-dimensional measurement 
device of the image pickup device 1 picks up an image of the head part of a person (a 
customer of a beauty salon or the like) as a target obj ect, several times from a different 
kinds direction, and perfonns predetennined image processing on the image, to obtain 
a three-dimensional shape of the face part (which is a part obtained by removing a hair 
part from the neck part). In case of a range finder, the neck part of the person as the 
target object is subjected to laser scanning, thereby measuring the distance,. to obtain . 
a three-dimensional shape of the neck part. The two-dimensional input device of the 
image pickup device 1 picks up an image of the neck part of the person as the target 
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object, to obtain a texture corresponding to the three-dimensional shape of the face 
part. 

In general, the three-dimensional shape measurement device of the image 
pickup device 1 cannot obtain three-dimensional shape data with respect to a part of 
dark color which absorbs light from an object, e.g., a part such as a head part having 
s . black hair or metal which reflects light. Therefore, the image pickup device 1 outputs 
|i three-dimensional shape data of the part such as the face or the like, which will be 

P3 hereinafter called a face part, i.e., the other part of the head part than the hair, and 

D 

yj texture image data associated with the three-dimensional shape data of the face cart 
W F ' 

Plural three-dimensional shape measurement devices and plural 

11 two-dimensional image input devices may be used for constructing the image pickup 

p device 1, in order to measure the neck part of the person as an object (which will be 

PJ 

described hereinafter as a customer) from many directions. ... 

• : The image processing device 2 is constructed by: a three-dimensional head part 
reconstruction processing section 3 for synthesizing three-dimensional shape data read 
from a head-part database 6 with three-dimensional shape data inputted from the 
image pickup device 1 , to construct a three-dimensional model of the neck part; a hair 
style mappingprocessing section 4 for mapping hair-style data read from the hair-style 
database 5 over the three-dimensional model of the neck part; a rendering section 5 for 
rendering the three-dimensional model of the neck part on which the hair-style data 
has been mapped, thereby to generate hair-style-matched image data; a head-part 
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database 6 in which three-dimensional shape data of a plurality of head parts are 
previously recorded; a hair-style database 7 in which a plurality of hair-style data items 
are previously recorded; and a controller 8 which is operated by a user to instruct 
selection of three-dimensional shape data of a head part stored in the head-part 
database 6, a hair-style data item stored in the hair-style database 7, or the like. 

The hair-style data stored in the hair-style database 7 is edited and generated 
with use of a three-dimensional graphic modeler or the like. Therefore, even a hair 
style which is actually difficult to make up can be generated and stored. The data 
structure of the hair-style database 7 will be described later with reference to FIG. 1 1 . 

The display device 9 is constructed by a LCD (Liquid Crystal Display) or CRT 
(Cathode Ray Tube) and the like, and displays a hair-style-matched image inputted 
from the image processing device 2. 

Next, the operation of the hair synthesis system will be explained with reference 
to the flowchart of FIG. 3 and FIG. 4. In the step SI, the image pickup device 1 picks 
up an image of the a neck part of a customer as a target object or laser-scans the neck 
part, to measure its three-dimensional shape, and outputs three-dimensional shape data 
and texture image data of the neck part of the customer to the image processing device 
2. 

The measurement method of measuring the three-dimensional shape may be any 
method, e.g., a method of using a stereo camera, a method of using a range finder, or 
the like. As described previously, however, the three-dimensional shape measurement 
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device of the image pickup device 1 cannot obtained three-dimensional shape data of 
the hair part. Therefore, three-dimensional shape data and texture image data (shown 
as measurement data in which a texture image is applied to three-dimensional shape 
data of the face part in FIG. 4) of the face part of the customer is outputted by the 
processing in the step S 1 . If plural three-dimensional shape measurement devices and 
two-dimensional image input devices are provided in the image pickup device 1, a 
plurality of sets each including three-dimensional shape data of the face part and 
texture image data are outputted. 

In the step S2, the three-dimensional head part reconstruction processing 
section 2 of the image processing device 2 constructs a three-dimensional shape model 
of a head part of a customer by using measurement data of the face part inputted from 
the image pickup device 1, i.e., the three-dimensional shape data and texture image 
data. . . - 

, Specifically, as shown in FIG. 4, head-part data selected and read from the 
head-part database 6 is combined with the measurement data of the face part of the 
customer, to reconstruct a perfect three-dimensional model of the head part of the. 
customer. The head-part database 6 is constructed by three-dimensional models of a 
plurality of head parts having arbitrary shapes. The head-part data is subjected to 
scaling and positional aligning, based on the size and direction in the measurement 
data of the face part of the customer, and is then combined with the measurement data 
of the customer. 
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The combination of the head-part data piece with the measurement data piece 
of the customer may generate a new polygon between open edges in both data pieces, 
or an open edge in either one data piece may be changed to fit an open edge of the 
other data piece. - 

. . The three-dimensional shape of the head part of the customer thus constructed 
u is expressed, for example, as a triangle-polygon model. In the three-dimensional 
| j . model of the head part of the customer thus constructed, hair is removed from the real 

63 image of the customer as shown in FIG. 4. 

O 

p -. . . In the step S3, operation of selecting hair-style data stored in the hair-style 

L database 7 by a user (the customer or the like) is received by. the controller 8. The 

u selected hair-style data is read o.utfrom the hair-style database 7, and is supplied to the 

!il 

|3 hair style mapping processing section 4. 

In this hair style selection, icons, thumbnails, or the. like indicating hair styles 
are displayed on- the display device 9 to invite the user to make a selection. The 
selection may be done by moving a cursor or the like on the screen of the display 
: device 9 or by inputting a number or the like designating a hair style. 

^ ■ In the step S4, the hair style mapping processing section 4 of the image 
processing device maps the supplied hair-style data on the three-dimensional mode of 
the head part of the customer supplied from the three-dimensional head part 
reconstruction processing section 3, and outputs the result to the rendering part 5. In 
the processing of mapping the hair-style data, hair data corresponding to each one of 
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hairs, which is included in the hair-style data, is mapped on the surface of the 
three-dimensional model of the head part, in accordance with the shape of the 
three-dimensional model of the head part, by the same manner as taken for texture 
mapping. Details of the mapping processing will be described later with reference to 
FIGS. 5 and 6. 

In the step S5, the rendering section 5 renders the three-dimensional model of 
the head part of the customer, on which the hair-style data has been mapped, to 
generate hair-style-matched image data as shown in FIG. 4, and outputs it to the 
display device 9. The method of rendering adopts texture mapping processing for a 
polygon model with respect to the face part of the threerdimensional model, and also 
rendering processing for a curvature model with respect to the hair part. The rendering 
processing for a curvature model is disclosed in details in, for example, "Simulation 
based on, a motion model of a string-like object and CG" by Kobayashi, Morishima, 
and Harashima, Shingakugihou, PRU90-127, pp. 15-20,199 L . - 

The display device 9 displays an image corresponding to the inputted 
hair-style-matched image data. The controller 8 invites a user (a customer or the like) 
to input whether or not the user is satisfied with the hair-style-matehed image 
displayed on the display device 9. 

.... At this time, the user can operate the controller 8 -to watch the 
hair-style-matched image while changing the viewpoint position and the illumination 
position. If the viewpoint position and the illumination position are changed, the 
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rendering section 5 executes again the rendering processing on the three-dimensional 
model of the head part and the hair-style data, so as to correspond to the changed 
condition (the viewpoint position and the illumination position) and outputs the result 
to the display device 9. 

In the step S5, if an input indicating that the user is not satisfied is received, the 
processing returns to the step S3, and the later processing is repeated. In the step S5, 
if.it is determined that an input indicating the user's satisfaction is received from the 
user, this processing is terminated. 

Next, explanation will be made of details of the hair-style mapping processing 
based on the hair-style mapping processing section 4. in the step S4. Beforehand, a 
structural example of the hair-style mapping processing section 4 will be explained 
with reference to FIG. 5. 

. The hair-style data read outsection 2 1 of the hair-style mapping processing 
section 4 reads hair-style data selected with use of the controller 8, from the hair-style 
database 7, and outputs it to the mapping position calculation section 22. The mapping 
position calculation section 22 calculates the position on the three-dimensional model 
where hair data (corresponding to a hair) forming part of the hair-style data should be 
mapped. A hair data interpolation processing section 23 obtains hair data to be 
mapped, by interpolation from the peripheral hair data around the hair-style data. The 
coordinate conversion section 24 maps the interpolated hair data by coordinate 
conversion from the object coordinate system of the hair data to the object coordinate 
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system of the three-dimensional model of the head part. 

Next, details of the hair style mapping processing of the hair style mapping 
processing section 4 will be explained with reference to the flowchart shown in FIG. 
6. The mapping position calculation section 22 of the hair style mapping processing 
section 4 has already been supplied with hair-style data from the hair-style data read 
outsection 21.. 

The hair-style data supplied to the mapping position calculation section 22 is 
obtained by two-dimensionally arranging hair data approximated to an object having 
a three-dimensional shape, as shown in FIG. 7. 

In the example of FIG. 7 A, every hair is approximated to a special curve such 
as a three-dimensional B-spline curve or the like. In the example of FIG. 7B, every 
hair is approximated to a polygon model. The addresses in the two-dimensional array 
correspond to mapping coordinate values supplied to the three-dimensional model of 
the head part on which the hair style is mapped, like the texture mapping processing. 

For example, if a mapping coordinate (u, v) = (0. 1 , 0. 1 ) is given to a vertex of 
a predetermined polygon of the three-dimensional model of the head part, the address 
on the hair-style data array of the hair data which is mapped on the vertex is (0. lu^, 
O-lVmax)' Where, the u max is the number of data pieces in the u-direction on the 
hair-style data array, and v max is the number of data pieces in the v-direction on the 
hair-style data array. , 

The hair-style data is defined by arranging the hair data having an arbitrary 
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three-dimensional shape into a two-dimensional array. Therefore, any arbitrary hair 
style can be expressed, and it is possible to generate hair-style data of a hair style 
which is difficult to cut. The method of generating hair-style data will be described 
later with reference to FIG. 16. 

At first, in the step S 11 , the mapping position calculation section 22 initializes 
the count value v in the longitudinal direction of the hair-style data array to 0.0. In the 
step S 12, the count value u in the lateral direction of the hair-style data array is 
initialized to 0.0. 

In the step S13, the mapping position calculation section 22 calculates the 
position on the three-dimensional model of the head part where hair data expressed by 
the address (u, v) of the hair-style data array is mapped. 

Explanation will now be made more specifically. FIG. 8 shows an arbitrary 
triangle polygon on the three-dimensional model of the head part. The vertexes A, B, 
and G of the triangle polygon respectively have attribute vectors A^, B artr , and C a11r 
which express attributes of the three-dimensional model of the head part. 
• The attribute is constructed by an object coordinate value of the 
three-dimensional model of the head part, local coordinate .axes at a vertex, and a 
mapping coordinate value of hair. Where vectors which form two arbitrary edges of 
the triangle polygon ABC are vectors VI and V2, the vectors VI and V2 can be , 
expressed by the next expression (1) with use of the attribute vectors of respective 
vertexes. 
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Vl=B a11r -A altr 

V2 = C attr -A at1r ...(1) 

Considering that the vectors VI and V2 are the coordinate axes of the triangle 
polygon ABC, an arbitrary point P h on the triangle polygon ABC can be expressed by 
the next expression (2). 

■ P h = A attr +aVl+bV2...(2) 

At this time, as shown in FIG. 8, coefficients a and b are defined in a manner 
that lengths between an arbitrary point P h and points at each of which a line parallel 
to one axis and extended from the arbitrary point crosses the other axis are normalized 
by the lengths of axial vectors, respectively. 

Where the mapping coordinate values of the vertexes A, B, and C are 
respectively (u a , v a ), (u b , v b ), and (u c , v c ), attention is paid to the point P h expressed by 
the expression (2), in relation to the elements of the mapping coordinate values. The 
next expression (3) then exists. 

' ■ ■■-■U h = U a + a (U b + U a ) + b (U c - U a ) 
• - - - V h = V a + a (V b - VJ + b (V c - V a ) ... (3) 

The expression (3) exists with respect to an arbitrary point in the (U, V) space. 

If the counter value u = u p and the counter value v = v p , the. condition for 
satisfying that the point P h on the three-dimensional model of the head part where hair . 
data should be mapped exists inside the triangle polygon is expressed by the next 
expression (4). 
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a + b< l...(4) 

That is, in the arbitrary triangle polygon of the three-dimensional model of the 
head part, if the coefficients a and b obtained by substituting u h = u p and v h = v p in the 
expression (3) satisfy the expression (4), the point P h exists inside the triangle polygon. . 
Therefore, the hair data expressed by (u p , v p ) on the hair-style data array is mapped on 
the triangle polygon. Accordingly, the mapping position of the hair data on the 
three-dimensional model of the hair data is calculated by using the expression (2) in 
the mapped triangle polygon. 

Returning to FIG. 6, in the step S14, the mapping position calculation section 
22 determines whether or not the triangle polygon of the three-dimensional model of 
the head part to be mapped is a hair generation region. This determination is carried 
out, based on a flag (positive or negative) which indicates whether or not hair data 
; added to each triangle polygon of the three-dimensional model of the head part should 
be mapped. 

If the mapping flag of the triangle polygon including the mapping position 
obtained by the processing in the step S13 is positive, the triangle polygon is 
determined to be a hair generation region, and the processing goes to the step S 1 5. . 

In the step SI 5, the hair data interpolation processing section 23 interpolates 
hair data,, which is expressed as (u p , v p ) and should be. mapped, with use of hair data 
stored in the hair-style data array. The interpolation method may use an average value 
of four close points or may use a summation average value of four close points. 
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Alternatively, the closest point may be directly used without interpolation. 

FIG. 9 shows a method of using a summation average value from four close 
points. Supposing now that the counter value u = u p and the counter value v = v p , the 
hairstyle data array address of the hair data to be mapped is P a = (m p , n p ) = (u p u, nBX5 
v p v max ). The hair data at the point P a is Hp, and the close 4 points are PI to P4 as 
shown in FIG. 9. The hair data pieces at the point PI to P4 are respectively H 1 to H4. 

, In the summation average method from 4 close points, the hair data Hp to be 
mapped is interpolated from the data of the 4 close points with use of the next 
expression (5). 

'-, Hp = dm(dnH2.+ (l-dn)Hl) + (l-dm)(dnH4 + (l-dn)H3 : ))... (5) .• 

If the hair data is a special curve such as a three-dimensional B-spline curve or 
the like as shown in FIG. 7 A, the interpolation processing of the hair data is obtained 
by interpolation of data of each control point. Meanwhile, if the hair data is a polygon 
model as shown in FIG. 7B, the interpolation processing of the hair data is obtained 
by interpolation between respective polygon vertex data pieces. In the expression (4), 
dm is the decimal part of nip, and dn is the decimal part of n p . 

Returning to FIG. 6, in the step SI 6, the coordinate conversion section 24 
coordinate converts hair data subjected to interpolation processing, from the object 
coordinate, system of the hair data into the object coordinate system of the 
three-dimensional model of the head part. 

The concept of the coordinate conversion will be explained with reference to 
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FIGS. 10A and 10B. Suppose that an arbitrary hair data piece on the hair-style data 
array is H. The hair data H is expressed as (I, J, K) according to the object coordinate 
system of the hair data. The object coordinate system (I, J, K) of the hair data may be 
arbitrarily given in case of a three-dimensional coordinate system in which the start 
point of the hair data H is an origin. For example, this is an orthogonal coordinate 
S3'stem in which the lateral direction of the hair-style data array is- 1, the longitudinal 

O direction thereof is J, and a direction perpendicular to these directions is K. Thus, the 

□ 

H start point of the hair data H is expressed as (I, J, K) = (0, 0, 0), and the other control 

ff, points are expressed by relative positions from the start point of the hair data. 

*"* Meanwhile, the three-dimensional model of the head part is expressed as an 

ll 

Lp object coordinate system (X, Y, Z). The object coordinate: system of the 

hk 

UT three-dimensional model of the head part is generally a three-dimensional orthogonal 

£3 

v - coordinate system* e.g. , an orthogonal coordinate system in which the direction toward 
the right ear from the gravity center position of the head part as an origin is X, the 
direction toward the top of the head is Y, and the direction toward the nose is Z. 

The coordinate conversion section 24 converts the hair data H expressed by the 
object coordinate system (I, J, K) of the hair data into a local coordinate system (T, S, . 
N) at the mapping position P h of the three-dimensional model of the head part obtained 
in the step S 13. The local coordinate system (T, S, N) at this point P h can be arbitrarily 
determined in case of the three-dimensional coordinate system using the point P h as 
an origin. For example, it is an orthogonal coordinate system in which N is the vector 
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in the normal line direction at the point P h , T is the vector in the tangent direction, 
which is obtained as a vector product of the vector in the radial direction toward the 
point P h from the center axis (Y-axis in FIG. 10) in the longitudinal direction of the 
. three-dimensional model of the head part and the vector N in the normal line direction, 
and S is the vector perpendicular to the normal line direction N and the tangent 
direction T. 

Suppose thatthe local coordinate system at each vertex of the three-dimensional 
model of the head part is previously given in the head-part database. Therefore, like 
the calculation of the mapping position, the local coordinate system (T, S, N) at the 
point P h is calculated with use of the expression (2) in the triangle polygon to be 
mapped. Supposing that data at the object coordinate system (I, J, K) of an arbitrary 
control point C ; of the hair data H is (i, j, k), the control point Q is expressed by the 
next expression (6) in the local coordinate system (T, S, N) at the point P h . 

Cy= i T+j S+kN ... (6) 

Where, the unit vectors of the local coordinate system (T, S, N) respectively 
have values of the object coordinate system (X, Y, Z) of the three-dimensional model 
of the head part. 

Therefore, the control point Q expressed by the expression (6) is expressed by 
the object coordinate system (X, Y, Z) of the three-dimensional model of the head 
part. 

Returning to FIG. 6, in the step SI 7, the mapping position calculation section 
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22 determines whether or not the counter value u exceeds 1.0. If the counter value u 
is not determined to exceed 1.0, the processing goes to the step SI 8. In the step SI 8, 
the mapping position calculation section 22 increase the counter value u by an 
increment value du. The increment value du indicates how long pitch hair data stored 
■ in the hair-style data array is mapped in the lateral direction. 

Thereafter, the processing returns to the step S 1 3, and the later processing is 
repeated. If it is determined in the step S 17 that the counter value u exceeds 1 .0, this 
means that mapping processing in the lateral direction reached the end of one line of 
the hair-style data array, so the processing goes to the step S 1 9. 

In the step S 1 9, the mapping position calculation section 22 determines whether 
- or not.the counter value v exceeds 1.0. If the counter value v is determined as not 
exceeding 1.0, the processing goes to the step S20. In the step S20, the mapping 
position calculation section 22 increases the counter value v by an increment value dv. 
The increment value dv indicates how long pitch hair data stored in the hair-style data 
array is mapped in the longitudinal direction. 

. * , Thereafter, the processing returns to the step S 12 and the later processing is 
repeated. In the step S19, if the counter value v is determined as exceeding L0, this 
means that mapping processing is completed with respect to all lines. The hair 
mapping processing is terminated, and the processing. goes to the step S5 shown in 
FIG. 3. 

In the step S14, if the mapping flag of the triangle polygon including the 
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mapping position obtained by the processing in the step S13 is negative, the triangle 
polygon is not determined to be a hair generation region, and the processing of the 
steps S15 and.S16 is skipped. 

By . thus specifying a hair generation region with use of a mapping flag, 
mapping can be effected only on an arbitrary region of the three-dimensional model 
of the head part, without mapping all the hair data pieces that construct thehair-style 
data read outfrom the hair-style database 7. Therefore, a hair-style such as a so-called 
Machican cut, a star cut, or the like can be easily generated. 

Also, according to this hair-style mapping processing, each of the hair-style data 
pieces stored in the hair-style database 7 is expressed by the object coordinate system 
of hair and can be mapped on a three-dimensional model of a head part having any 
arbitrary shape, without depending on the shape of the human head part. 

Next, , the data structure of the hair-style database 7 will be explained with 
reference to FIG. 11. The hair-style database 7 is constructed by a plurality of 
hair-style data pieces. Each hair-style data piece is of a two-dimensional array as 
described above, and at the top, the number of data pieces in the lateral direction 
and the number of data pieces v max in the longitudinal direction. Following them, hair 
data pieces for (u^ x Vmax ) are described. 

In some cases, group information (described later) is described at the top of the , 
hair data piece. Subsequently, a plurality of control point data pieces are described, 
in case where hair data is approximated to a three-dimensional parametric curve or the 
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like. The control point data pieces are expressed by the hair object coordinate system 
(U,K). 

The hair data may be three-dimensional polygon data. In this case, a plurality 
of polygon vertex data pieces are described as hair data. 

The group information described in hair data will be explained now. The group 
information can be used for hair data interpolation processing in the step SI 5 of the 
hair-style mapping processing (FIG. 6) described above. 

With reference to FIG. 12, explanation will be made of the hair data 
interpolation processing in case where group infonnation is supplied. Where the 
counter value u ■=• u p and the counter value v = v p are given,, the address on the hair- 
style data array of hair data to be mapped is P a = (nip, n p ) = (Up u^x, v p v ma)i ). 

Suppose that the hair data at the point P a is Hp, four close points are points P 1 
to P4 as shown. in- FIG. 12, and hair, data pieces at the points PI to P4 are respectively 
H 1 to H4 . Further, 1 is described as group infonnation in the hair data piece H 1 , and 
2 is described as group infonnation in each of the hair data pieces H2 to H4 . That is, 
in this example, only the hair data piece HI belongs to a group 1, and the other hair 
data pieces H2 to H4 belong to a group 2. If group infonnation is given, only the hair 
data pieces that belong to one same group are used to perfonn interpolation, in the hair 
data interpolation processing. ...... 

More specifically, among the four close points of the point P^ which is mapped 
at first, the closest point if obtained and the group infonnation thereof is obtained. In 
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case of FIG. 12, the closest point is the point P3. Next, among the four close points 
PI to P4, the number of close points that belong to the same group as that of the 
closest point is obtained. 

If no other point among the four close points PI to P4 than the close point 
belongs to the same group as that of the closest point, the hair data piece of the closest 
point is directly used also for the point P a . .. 

If two close points including the closest point among the four close points PI 
to P4 belong to the same group as that of the closest point, a linear interpolation value 
between the two points is used for interpolation. 

• If three close points including the closest point among the four close points PI 
to P4 belong to the same group as that of the closest point, i.e., in the case shown in 
FIG. 12, an interpolation value is calculated by the next expression (7). 

; , Hp-(l-r)H32 + rH34 
m. i H32=dn H2 + (l-dn)H3 
H34 = dnH4 + (l-dn)H3 
r = (m p - m32)/(m34 - m32) ... (7) 

In this expression, dn is the decimal part of n p , m32 is the m-address at the point 
P32, and m34 is the m-address at the point P34. 

By thus describing group information in hair data, for example, it is possible to 
express even a hair style having parting of hair where interpolation will cause a 
problem. That is, if grouping at the boundary of parting is performed, hair data pieces 
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in both sides of the boundary can be used to prevent generation of unnatural hair, in 
case where four close points spread over the boundary. 

Next, with reference to FIGS. 12 and 13, explanation will be made of an 
applicative example of the coordinate conversion in the processing in the step SI 6 in 
the hair-style mapping processing (FIG. 6) described above. 

In the coordinate conversion processing in the step SI 6, unevenness in a hair 
style can be expressed with use of random numbers. FIG. 13A shows a state where 
unevenness in ahair style is not expressed while FIG. 13B shows an example of a state 
in which unevenness in a hair style is expressed by rotating the local coordinate system 
with use of a random number in the coordinate conversion. 

Specifically, the local coordinate system (T, S, N) of the head part is rotated 
about an axis in correspondence with a generated random number where the axis is N. 
Thereafter, the result is converted into the object coordinate system (X, Y, Z) of the 
head part model. By this processing, the directions in which respective hairs grow can 
be changed delicately, without changing the positions where hair data should be 
mapped. 

i FIG. 1 4 A shows a state in which unevenness in a hair style is not expressed, 
while FIG. 14B shows an example in which unevenness in a hair style is expressed by 
changing mapping positions by random numbers in the. coordinate conversion. 

Specifically, the position of P h as the origin of the local coordinate system (T, 
S, N) is shifted to P„ in correspondence with a generated random number, and is 
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thereafter converted into the object coordinate system (X, Y, Z) of the head part 
model. By this processing, the position where the hair grows can be changed 
delicately without changing the hair style. 

The technique of rotating the axis of the coordinate system to be converted use - 
the random number and/or moving its origin to express natural unevenness in a 
synthesized . image can be applied to other cases than the synthesis of hair images. 

Meanwhile,. in this hair synthesis system, hair-style data can be mapped after 
changing the three-dimensional model of the head part into an arbitrary shape by the 
three-dimensional head part reconstruction processing section 3. FIG. 15A shows a 
state in which the three-dimensional model of a head part is not changed, while FIG. . 
1 5B shows a state in which the three-dimensional model of the head part is magnified 
1.5 times in the Y-axis direction, 
.v Magnification/minification processing for a three-dimensional model of a head 
part is achieved by affine conversion processing. At this time, similar affine 
conversion processing is performed also on the local coordinate system (T, S ; N) 
assigned to an arbitrary polygon vertex of the three-dimensional model of a head part, 
: As shown in FIG. 1 5B, in case of magnifying the three-dimensional model of 
the head part for 1 .5 times in the Y-axis direction, vectors of T, S, and N are multiplied 
by- a conversion matrix for 1.5 times magnification in the Y-axis, direction, to obtain 
a local coordinate system (T', S', N') expanded in the longitudinal direction. By 
mapping hair data H on the local coordinate system (T, S', N') thus deformed, the hair 



31 

style is magnified 1.5 times in the Y-axis direction, into a shape matched with the head 
part. By this processing, same hair-style data can be mapped on the three-dimensional 
model of a head part, which is deformed into an arbitrary shape. Next, generation 
of hair-style data will be explained with reference to FIG. 16. FIG. 16 shows a 
columnar, coordinate system which surrounds a head part. In the columnar coordinate 
system, the axis penetrating the center of a column is taken as the Y-axis, the distance 
from the Y-axis to a plane perpendicular to the Y-axis is r, and the angle from a 

reference direction on this plane is 0. 

An arbitrary point P h on the surface of the head part can be expressed as P h = 

(y p , r^, 0 p ) with use of the columnar coordinate system. Further, if the line defined by 
: Y-5=,y/jp and 6 = 6 p of the columnar coordinate system always has one contact point 

with the surface of the head part, the arbitrary point P h on the surface of the head part 

can be expressed by (y p , 0 p ). . .. 

-■, -s That is, if the shape ofthe surface of the head part is always convex toward the 

circumferential direction, the above condition (the line defined by Y= y p and 0 = 0 p 

ofthe columnar coordinate system always has one contact point with the surface ofthe 

head part) is satisfied. 

Considering the scalp part excluding the face, the shape of the surface of the 

head part is generally considered to be always convex toward the circumferential 

direction. Therefore, the arbitrary point P h on the scalp part ofthe head-part model 

can be expressed by P h = (y p , 0 p ). 
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Creation of hair-style data can be relatively easily carried out by using a system 
disclosed in "Development of a hair design system using tuft models" by Kishi, 
Saegusa, and Morishima, in Shingakugihou, MVE97-102, pp. 67/74, 1998. In the 
system, hair-style data is prepared by editing hair data generated on a given head-part 
model. Hair data outputted from this system is expressed by the object coordinate 
system, of the given head-part model. 

The hair database 7 in the present invention records, hair-style data which is 
stored in a two-dimensional array by mapping the hair data on a space expressed by 
(y, 6) of a columnar coordinate system. For example, the following expression (8) is 
used for conversion from the (y, 6) space into the (u,v) space of the two-dimensional 
. array as shown in FIG. 7. 

u=e p /27r 

v = (y p -y min )/(y max -y min )... (8) 

In this expression, y min is the smallest value of the Y-axis of the columnar 
coordinate system of the surface of the head part, and y max is the largest value of the 
Y-axis of the columnar coordinate system of the surface of the head part. 

The conversion from the (y, 0) space into the (u, v) space may be of non-linear. 
For example, sampling near the top part of the head can be prevented from being 
coarse, as long as the conversion is arranged such that the increment in the v-direction 
becomes smaller as it comes closer to the head top. 

As described above, hair-style data consisting of hair data converted into the (u, 
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v) space is recorded in the hair-style database 7. 

The present invention is also applicable to the case of expressing a string-like 
object other than hair in CG. 

; - Meanwhile, the series of processing in the image processing apparatus 2 as 
described above can be executed by both hardware and software. In case of executing 
the series of processing by software, the software can.be installed from a recording 
medium into a computer in which the program constituting the . software is 
incorporated in dedicated hardware or a general-purpose computer which can execute 
various functions computer by installing various programs. 

FIG: 17 shows a structural example of a personal computer which realizes the 
image processing apparatus 2. This personal computer includes a CPU (Central 
Processing Unit) 41. The CPU 41 is connected with an input/output interface 45 
through a bus 44. The input/output interface 45 is connected with: an input section 46 
constructed by, an input device such as a keyboard, a mouse, or the like; an image 
output section 47 for outputting a hair-style-matched image to the display device 9; a 
storage section 48 constructed by a hard-disc drive for storing a program, various data, 
and the like; a communication section 49 for communicating data through Internet or 
the like; and a drive 50 for reading/writing data with respect to a recording medium 
such as a magnetic disc 5 1, a semiconductor memory 54, or. the like. The bus 44 is 
connected with a ROM (Read Only Memory) 42 and a RAM (Random Access 
Memory) 43. 
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An image processing program which lets this personal computer execute 
operation as the image processing apparatus 2 is supplied in the form that it is stored 
in the magnetic disc 5 1 or semiconductor memory 54, to the personal computer. It is 
read by the drive 50 and installed in the hard disc drive included in the storage section 
48. The image processing program installed in the storage section 48 is loaded from 

E the storage section 48 to the RAM 43 and executed, in accordance with instructions 

jp from the GPU 41 corresponding to commands from the user. 

13 In the present specification, the step of describing the program recorded in the 

P 

I j recording medium includes not only processing which is carried out based on time in 

L accordance with a described order but also processing which is executed in parallel or 

HI 

u individually although it is not always performed based on time. 

in 

O Also, in the present specification, the term of "system" means the entire 

PJ 

apparatus constructed by a plurality of apparatuses and/or devices. 



Industrial Applicability 

; ; As described above, according to the present invention, the hair-style is stored, 
stored hair-style data is read out, and hair data corresponding to a hair and included 
in the hair-style data is mapped at a predetermined position of a three-dimensional 
shape image of a head part. . Therefore, a hair style consisting of hairs every of which 
is expressed by CG can be changed so as to correspond to plural different head shapes. 
Also, according to the present invention, stored hair-style data is read out, and 
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hair data corresponding to a hair and included in the read outhair-style data is mapped 
at a predetermined position on a three-dimensional shape image of a head part. Based 
on the mapping results, a hair-style-matched image is generated. Therefore, a hair 
style consisting of hairs every of which is expressed by CG can be changed so as to 
correspond to plural different head shapes. 



